1. Field of the Invention
The present invention relates to Web applications. More particularly, the present invention relates to creating extensible and scalable Web applications.
2. Description of the Related Art
Web servers and Web applications constitute the single most important component of the network content delivery system we know as “The Web”. The first Web Servers started to appear in 1994 on systems running the UNIX™ operating system (OS) or a UNIX-based OS on the Internet. The design of those early systems reflects its UNIX-based heritage. URLs (Uniform Resource Locators) are equivalent to UNIX-based OS file names. Each URL, when requested by a client program, typically a “Web Browser”, is mapped to a file of the same name, wrapped in the HTTP (HyperText Transport Protocol) and delivered to the client. In those cases where the content can not be represented as a static file and needs to be dynamically generated, the file representing the URL of the client request specifies the program that is run to generate the content. This capability, known as CGI (Common Gateway Interface) stems from the traditional UNIX-based OS designers' practice of making everything look like a file. Thus each URL represents a file that either contains the content, or contains the program that is used to generate the content.
Over the next five years the Web saw explosive growth, and the architecture of the original Web servers, though simple and elegant, was beginning to strain. Static content was still delivered by mapping URL's into files, but dynamic content was becoming problematic. The notion of programs as files, as well as the mechanisms for launching, managing, and communicating with CGI programs is very specific to a UNIX-based OS, which makes porting Web servers and their corresponding content to a non UNIX-based OS difficult. In addition, as content management techniques required more of the content to be generated dynamically, even if simply to paste together several static files as a single URL, the CGI programs rapidly became the bottle neck. Each dynamic page requires a separate program to be launched and executed by the operating system, only to be terminated each time a request was completed. Additionally, the communication between the Web Server and the CGI program is very limited. Only the URL and its corresponding HTTP envelope information is made available to the CGI program, which can only return content; the ability to pass meta-information back to the server is almost non-existent.
The next state in the evolution of Web servers focused on eliminating the CGI bottle neck, and specifically the program creation and execution step required for each URL requested. Generally, three different approaches have been taken: keeping the basic CGI interface, only making it faster; building Web server specific APIs (Application Programming Interfaces) by requiring the dynamic code generating portions to be bound into the same process as the Web server; or defining language specific APIs whose implementations do not require the overhead implied by the CGI model.
The Fast CGI interface tries to improve the performance of the CGI specification by eliminating the process creation and execution step at every request, yet maintaining backward compatibility wherever possible. The Fast CGI interface, represented by the file that maps from the URL, is created and started once when the Web server starts. Multiple independent URL requests are sent to the same fast CGI process by defining a request packet protocol than can accommodate multiple requests and responses for each Fast CGI process. Fast-CGI has the advantage of preserving a separate execution context for dynamic content generation, while eliminating the bulk of the process creation overhead of traditional CGI programs. Consequently, Fast CGI programs are easily ported to work with many different Web servers.
The second approach to eliminating the CGI bottleneck is to move the dynamic content generation program into the same execution context as the server, by expressing the dynamic content generation in terms of APIs that are specific to a particular Web server. This approach eliminates the process creation and execution overhead of CGI programs entirely, but at the expense of close coupling to a particular Web server. Most major Web servers provide such API definitions. However, dynamic content generation using these APIs is rarely portable to a different server. In addition, by having the dynamic content generation in the same execution context as the server, a bug in a dynamic content generation module can negatively impact the entire Web server, including URL requests that have nothing to do with the bug-containing module.
The third approach used to eliminate the CGI bottleneck is to create a set of language specific APIs that can be logically bound into the execution context of the Web server, yet be defined in a Web server independent way. Servlets are the leading example of this approach. A servlet may be a JAVA™ programming language module that conforms to a defined set of language-specific APIs, which can be (and have been) implemented to provide dynamic content for many different Web servers. Thus servlets combine the advantages of Fast CGI portability to different Web servers—with the close coupling of server specific extensions.
Although all three approaches reduce performance problems associated with the CGI interface, they still fundamentally retain the notion of a one-to-one mapping between URLs and files, where all requests and their corresponding files are completely independent.
As the Web has grown, the notion that every URL request and its associated file is independent of any other request has become a serious architectural roadblock. It is now common for a single Web “form” to be spread over multiple pages (URLs), or for a single user to have unique state associated with a sequence of requests that may span days or even years. Finally, as the sheer volume of content on the Web has mushroomed, it is often no longer appropriate to assume, as is implicit in the CGI one-file-per-URL model, that the content resides on the server machine at all. The software architecture that was designed to deliver individual pages in response to URL requests is now being used to build sophisticated applications, whose content happens to be wrapped in HTTP. Somewhere in the switch from delivering static files as URLs to creating full blown applications, Web servers became Web application development frameworks.
As the need for more sophisticated features has grown, so too have the capabilities of the Web servers used to implement them. However, they are still based on the original one file per request architecture that was seemingly previously advantageous, but now burdensome. To support these added capabilities, the size and complexity of the APIs has grown. The descendants of the CGI architecture are stressed to provide functionality that isn't a good fit for their designs. As an example, a recent Servlet API (2.0) needs over two dozen classes and almost ten times that many methods to describe its interface.
The entire reason for the explosion of interface complexity isn't completely due to the complexity of the interactions required by implementors of the interface. As Web servers have become Web application frameworks, the notion that the same set of content can be delivered by any server has persisted. Somehow the “content” is viewed as separable from the server used to deliver it. Consequently, every new Web server designer feels obliged to incorporate every nuance and capability of every previously deployed server, to insure that pre-existing content can be delivered with the new software without change.
As the Web matures, a transition is occurring away from the current client-server paradigm, where browsers are pointed at particular Web sites, whose servers deliver all the content on that site, to a more distributed model. In this new model, both the traditional browsers and servers will still exist, but the content received by the client for a particular page is likely to have been retrieved from several traditional back-end servers, and modified to suit the requirements and desires of the particular client. This is a “work flow” model, where the content passes through various stages, and is transformed at each stage.
Early versions of these intermediate stages, termed meta-servers, are already starting to appear on the Web. Some of the meta-servers are called “portals”, and others are known as “content aggregators”. For the most part, portals and content aggregators are one in the same. It is a portal when viewed from the perspective of the client, and a content aggregator from the perspective of the traditional, or “content”, server.
As these meta-servers begin to play a more prominent role in the infrastructure, they will have a profound impact on the way in which traditional content servers are constructed. No longer will the content server produce both the content and its presentation (look and feel). Meta-servers will transform the content after it leaves the content server, allowing content servers to be simpler. Today's content servers not only provide the content, but manage the presentation, user preferences, and browser differences as well.